Printer user interface

ABSTRACT

A multifunction printer enables the printing of time-based media and is operated via a user interface. The user interface includes a touch screen for accepting command inputs and providing information to a user. Time-based media data is received by the printer from a media source specified via the user interface. A user specifies one or more multimedia processing functions for the printer to apply to the data. The printer performs the specified functions, and previews the output to the user via a display of the user interface. If the user decides to print the previewed output, the user specifies one or more output devices, such that the printer can print conventional printed output and/or to a specified electronic output.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the following provisional patentapplications, each of which is incorporated by reference in itsentirety: U.S. Provisional Application No. 60/506,206, filed Sep. 25,2003; U.S. Provisional Application No. 60/506,263, filed Sep. 25, 2003;U.S. Provisional Application No. 60/506,302, filed Sep. 25, 2003; U.S.Provisional Application No. 60/506,303, filed Sep. 25, 2003; and U.S.Provisional Application No. 60/506,411, filed Sep. 25, 2003.

This application is also related to the following applications, each ofwhich was filed on Mar. 30, 2004, and is incorporated by referenceherein in its entirety: application Ser. No. 10/814,931, entitled“Printer Having Embedded Functionality For Printing Time-Based Media”,application Ser. No. 10/814,500 entitled, “User Interface for NetworkedPrinter,” application Ser. No. 10/814,845 entitled, “Stand AloneMultimedia Printer With User Interface for Allocating Processing,”application Ser. No. 10/814,842 entitled, “Printer with MultimediaServer,” and application Ser. No. 10/814,944 entitled, “Multimedia PrintDriver Dialog Interfaces,”.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates generally to printers that have embeddedfunctionality for printing time-based media, and in particular to a userinterface for operating a printer that prints time-based media resultingin a combination of a printed output and a related electronic dataoutput.

2. Description of the Related Art

Conventional printers receive documents in a variety of formats andprint the contents of the documents in accordance with a proper format.For example, a printer enabled to print Portable Document Format (PDF)documents will correctly recreate the original appearance of thedocument, regardless of the platform being used to view the document.

Today, as more databases and computer networks are interconnected,people often have multiple data systems and destinations in which tostore information. For example, a person may receive an email containinginformation that he wants to retain. The person may want to print someor all of the information. The person may further want to add theinformation to a database or to send the information to other people ordestinations or to add the information to a web page. Currently, theperson will need to execute several different software programs and willneed to type multiple commands into the programs. He may also need tore-enter the information into one or more programs. This is notefficient and is prone to human error, since human beings occasionallyforget to perform one of more of the tasks usually associated with areceived document and are also prone to making typographical errors.

Some conventional printers incorporate a management function in whichthe printer monitors its own internal functions and displays an alertif, for example, its toner is low or it is out of paper. This action isbased on the printer doing “self-monitoring,” not on any monitoring ofthe documents to be printed.

While conventional printers can print documents in a wide variety offormats, these printers are fundamentally limited in their ability toreproduce different kinds of media. For example, it is standardtechnology for a printer to produce images of static text, pictures, ora combination of the two. But because these printers print onto paper oranother similar fixed medium, they cannot record the nuances oftime-based media very well.

In developing a printer that is equipped to print time-based mediawithout the limitations of conventional printers, a user interface forsuch a printer is needed. It is further desirable that such a userinterface be operable with a printer that performs at least some of thenecessary processing itself rather than require an attached computer orother device to perform all of the processing.

SUMMARY OF THE INVENTION

A multifunction printer enables the printing of time-based media. In atypical hardware configuration for such a multifunction printer, aprinter includes a print engine that produces a paper or other printedoutput and one or more electronic devices the produce a relatedelectronic output. Together, the printed and electronic outputs providean improved representation of the time-based media over that of aconvention paper printer.

A user interface provides access to functionality of the printer. In apreferred embodiment, the user interface includes a touch screen foraccepting command inputs and providing information to a user. In analternative embodiment, input is provided by way of a keypad, keyboard,or other input device.

Time-based media data is received by the printer from a media sourcespecified via the user interface. A user specifies one or moremultimedia processing functions for the printer to apply to the data.The printer performs the specified functions, and previews the output tothe user via a display of the user interface. If the user decides toprint the previewed output, the user specifies one or more outputdevices, such that the printer can print conventional printed outputand/or to a specified electronic output.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a printer having a user interfacein accordance with an embodiment of the present invention.

FIG. 2 illustrates an overview of a generalized process for creating arepresentation of time-based media data in accordance with an embodimentof the present invention.

FIG. 3 illustrates an example of a user interface for selecting an inputsource in accordance with an embodiment of the present invention.

FIG. 4 illustrates a user interface display for selecting an audiomultimedia processing function in accordance with an embodiment of thepresent invention.

FIG. 5 illustrates a user interface display for selecting a videomultimedia processing function in accordance with an embodiment of thepresent invention.

FIG. 6 illustrates a user interface display for previewing output inaccordance with an embodiment of the present invention.

FIG. 7 illustrates a user interface display for selecting output devicesfor printing in accordance with an embodiment of the present invention.

The figures depict preferred embodiments of the present invention forpurposes of illustration only. One skilled in the art will readilyrecognize from the following discussion that alternative embodiments ofthe structures and methods illustrated herein may be employed withoutdeparting from the principles of the invention described herein.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Various embodiments of a multifunction printer enable the printing oftime-based media in a useful and intelligent format. To create arepresentation of this time-based media, the printer produces a printedoutput and a related electronic output, which together provide arepresentation of the received media. Depending on the desiredapplication for the printer, the printer may include any number ofdevices for receiving the media, printing the printed output, andproducing the electronic output. The described user interface is meantto be generally applicable not only to the specific combinations ofinput and output devices described herein, but also to additionalcombinations not specifically disclosed, as would be understood by thoseof skill in the art from the present disclosure.

Printer Architecture

FIG. 1 is a high-level diagram of one embodiment of the printer 100.Printer 100 includes a media source interface 105, a user interface 110,a printed output system 115, an electronic output system 120, and amedia processing system 125. Capable of receiving time-based media 150,the media source interface 105 can take a variety of forms and mayinclude one or more devices that can receive media data or create mediadata by observing a media event. Similarly, the printed output system115 and the electronic output system 120 can take a variety of forms andmay each include one or more devices tat can produce, respectively, aprinted output 160 and an electronic output 170.

In one embodiment, the media processing system 125 includes a memory130, a processor 135, and one or more embedded functionality modules140. The embedded functionality modules 140 may include software,hardware, or a combination thereof for implementing at least a portionof the functionality of the multifunction printer 100. The mediaprocessing system 125 is coupled to the media source interface 105 andthe user interface 110, allowing it to communicate with each of thosedevices. The media processing system 125 is also coupled to the printedoutput system 115 and to the electronic output system 120 for providingthe appropriate commands and data to those systems.

Printer 100 is further described in related application, “Printer HavingEmbedded Functionality for Printing Time-Based Media,” filed Mar. 30,2004, Ser. No. 10/814,931, which application is incorporated byreference in its entirety

FIG. 2 illustrates an overview of a generalized process in which theprinter 100 creates a representation of time-based media data 150, inaccordance with one embodiment of the invention. First in theillustrated embodiment, time-based media data is received 202 by theprinter 100. Using UI 110, a user selects 204 a processing function tobe performed on the data. Printer 100 then performs 206 the specifiedprocessing function, and previews 208 the output for the user. If theuser is satisfied with the data as previewed, he selects 210 one or moreoutput devices to which to send the time-based media data, and finallyinstructs printer 100 to output 212 the data. Each of the above steps isnow described in greater detail.

User interface (UI) 110 of printer 100 preferably includes a screen onwhich information can be conveyed to the user. UI 110 preferably alsoincludes a mechanism for allowing a user to input responses and makeselections. In a preferred embodiment, printer UI 110 includes a touchscreen, and the user makes selections and inputs responses by touchingthe appropriate part of the screen. In one embodiment, a keypad is alsoprovided for entry of alphanumeric data. In an alternative embodiment, ajoystick, trackball or other input device can be used to provide inputand make selections.

FIG. 3 illustrates an example of a user interface 110 for selecting aninput source. Display 302 of UI 110 includes an instruction to the userto “select input source”, and several selections for the user to choosefrom. In the illustrated embodiment, the user can select from CD 304,DVD 306, VCR 308 Line In 310, Direct Memory Device 318 and Bluetoothdevice 320. Those of skill in the art will appreciate that in otherembodiments, other input sources will be available. Also shown is aCancel button 312, should the user wish to cancel the action ofselecting an input source, and a Continue button 314 for finalizing theselection. In a preferred embodiment, the user makes a selection bytouching the appropriate button on the screen. In an alternativeembodiment, the user can make the selection by pressing an appropriatebutton on keypad 316.

Once a data source has been specified so that data can be received 202,a multimedia processing function is selected. In a preferred embodiment,processing functions include audio range selection and video rangeselection.

Audio Range Selection

In a preferred embodiment, printer 100 is capable of performing manykinds of multimedia processing functions on the audio selection.Available functions preferably include event detection, speakersegmentation, speaker recognition, sound source location, speechrecognition and profile analysis. As can be seen in FIG. 4, userinterface 110 provides a user with the ability to select from amongthese options. In FIG. 4, display 302 of UI 110 includes a userinstruction 402, instructing the user to select the desired multimediaprocessing function. Display 302 also includes the available options,which in the illustrated case are print segment 403, event detection404, sound source locator 406, speaker recognition 408, profile analysis410, speech recognition 412, and speaker segmentation 414. A button 318is also provided in one embodiment to allow the current input to bechanged, as described above with respect to FIG. 3. Finally, a cancelbutton 312 is available to allow the user to back to a previous screen,and a continue button 314 is available to be selected when the user hasspecified all of the desired processing functions.

Selecting one of the multimedia processing functions displays a submenufor the user specific to that processing function.

In one embodiment, if a user selects the Print Segment button 403 ondisplay 302, a dialog appears on display 302 showing an audio waveformand additionally providing slider controls with which the user canselect portions of the current input to print. The manner in whichoutput is created by the printer is discussed below.

Selecting the Event Detection button 404 displays a dialog to the usershowing the results of applying audio event detection, such as clapping,yelling, or laughing, along a timeline. Each detected event ispreferably accompanied by a confidence that it was detected correctly.The display 302 also includes a slider that lets the user adjust athreshold on the confidence values. As the slider is moved, events thatare above threshold are displayed. The audio file is then preferablysegmented into clips that bound the above-threshold events. In oneembodiment, the display 302 also includes a series of check boxes thatlet the user choose which events to display.

Selecting the Speaker Segmentation button 414 presents a dialog showingthe results of applying speaker segmentation along a timeline. Eachsegment is preferably shown in a different color and segments that wereproduced by the same speaker are shown in the same color.

Selecting the Speaker Recognition button 408 displays a dialog showingthe results of applying speaker recognition along a timeline. In apreferred embodiment, the name of each speaker is accompanied by aconfidence that it was detected correctly. The dialog preferablyincludes a series of check boxes that let the user choose which speakersto display, as well as a slider that lets the user adjust a threshold onthe confidence values. As the slider is moved, speaker names that areabove threshold are displayed. The audio file is segmented into clipsthat bound the above-threshold speaker identities.

In another embodiment, Speaker Segmentation and Speaker Recognitionfunctions can be combined. In such an embodiment, a dialog shows theresults of applying speaker segmentation and speaker recognition along atimeline. Each segment is shown in a different color and segments thatwere produced by the same speaker are shown in the same color. Eachsegment is accompanied by a confidence that the segmentation is correct.The speaker recognition results include text and optionally confidencevalues for each speaker name. Multiple speaker names could be associatedwith each segment. The user interface includes sliders that let the useradjust thresholds on confidence values. As the sliders are moved,speaker segments or speaker recognition results that are “abovethreshold” are displayed.

Selecting the Sound Source Locator button 406 displays a dialog showingthe results of applying sound source localization along a timeline. Thedirection from which sound was detected is displayed as a sector of acircle. Each sector is accompanied by a confidence that it was detectedcorrectly. The dialog additionally preferably includes a series of checkboxes arranged around the circumference of a prototype circle that letthe user choose which directions to display.

In another embodiment, Event Detection and Sound Source Locatorfunctions can be combined. In such an embodiment, a dialog shows theresults of applying sound source localization and audio event detectionalong a timeline. The direction from which sound was detected isdisplayed as a sector of a circle. Each sector is accompanied by aconfidence that it was detected correctly. The user interface includes aseries of check boxes arranged around the circumference of a prototypecircle that let the user choose which directions to display. Eachdetected event is accompanied by a confidence that it was detectedcorrectly. The user interface includes a series of check boxes that letthe user choose which events (e.g., clapping, yelling, laughing) todisplay.

If a user selects the Speech Recognition button 412, a dialog shows theresults of applying speech recognition along a timeline. This includestext and optionally confidence values for each word or sentence. Thedialog preferably includes a slider that lets the user adjust athreshold on the confidence values. As the slider is moved, words thatare above threshold are displayed. The audio file is segmented intoclips that bound the above-threshold words.

In another embodiment, the speech recognition results are matchedagainst results from a Profile Analysis 410, in which a pre-existingtext-based profile that represents the user's interests. The dialogincludes a slider that lets the user adjust a threshold on theconfidence values. Another slider adjusts a threshold on the degree ofmatch between the profile and the speech recognition results. As thesliders are moved, words that are above threshold are displayed.

In another embodiment, the user interface also shows the results ofapplying audio event detection, such as clapping, yelling, or laughing,along a timeline in combination with speech recognition, speakersegmentation, or speaker recognition. Each detected event is accompaniedby a confidence that it was detected correctly. The user interfaceincludes sliders that let the user adjust thresholds on the confidencevalues.

In another embodiment, the functions of Event Detection, SpeechRecognition and Speaker Recognition can be combined. In such anembodiment, a dialog shows the results of applying speech recognitionalong a timeline, and additionally shows the results of applying audioevent detection, such as clapping, yelling, or laughing, along atimeline; and the results of applying speaker recognition along thetimeline. The dialog includes a series of check boxes that let the userchoose which speakers to display, and additionally includes sliders thatlet the user adjust thresholds on the confidence values.

In another embodiment, Event Detection, Speech Recognition and SpeakerSegmentation can be combined. In such an embodiment, a dialog shows theresults of applying speech recognition along a timeline, andadditionally shows the results of applying audio event detection, suchas clapping, yelling, or laughing, along a timeline; and the results ofapplying speaker segmentation along the timeline. Each segment is shownin a different color and segments that were produced by the same speakerare shown in the same color. The dialog includes sliders that let theuser adjust thresholds on the confidence values.

In another embodiment, Speech Recognition, Event Detection and SoundSource Locator functions can be combined. In such an embodiment, adialog shows the results of applying speech recognition along atimeline, and additionally shows the results of applying audio eventdetection, such as clapping, yelling, or laughing, along a timeline, andalso displays the direction from which sound was detected as a sector ofa circle. The dialog includes a series of check boxes arranged aroundthe circumference of a prototype circle that let the user choose whichdirections to display. The dialog includes sliders that let the useradjust thresholds on the confidence values.

Video Range Selection

In a preferred embodiment, printer 100 is capable of performing manymultimedia processing functions on a video selection. Availablefunctions preferably include event detection, color histogram analysis,face detection, face recognition, optical character recognition (OCR),motion analysis, distance estimation, foreground/backgroundsegmentation, scene segmentation, automobile recognition and licenseplate recognition. As can be seen in FIG. 5, user interface 110 providesa user with the ability to select from among these options. In FIG. 5,display 302 of UI 110 includes a user instruction 502, instructing theuser to select the desired multimedia processing function. Display 302also includes the available options, which in the illustrated case areselectable via buttons for Print Segment 503, Event Detection 524, ColorHistogram Analysis 526, Face Detection 504, Face Recognition 522,Optical Character Recognition (OCR) 512, Motion Analysis 506, DistanceEstimation 510, Foreground/Background Segmentation 514, SceneSegmentation 518, Automobile Recognition 516 and license platerecognition 520. A button 418 is also provided in one embodiment toallow the current input to be changed, as described above with respectto FIG. 3. Finally, a cancel button 312 is available that returns theuser to a previous screen, and a continue button 314 is available to beselected when the user has specified all of the desired processingfunctions.

Selecting one of the multimedia processing functions displays a submenufor the user specific to that processing function.

In one embodiment, if a user selects the Print Segment button 403,dialog box 302 shows key frames along a timeline and has slider controlsthat let the user select portions of a given video file to print. Themanner in which output is created by the printer is discussed below.

Selecting the Event Detection button 524 displays a dialog showing theresults of applying a video event detection algorithm along a timeline.Examples of video events include cases when people stand up during ameeting or when they enter a room. A slider control is preferablyprovided to let the user select portions of a given video file to print,based on a confidence value.

Selecting the Color Histogram button 526 presents a dialog showing theresults of applying a color histogram analysis algorithm along atimeline. For example, a hue diagram is shown at every 30-secondinterval. This allows a user to quickly locate the portions of a videothat contain sunsets, for example. A slider control is preferablyprovided that lets the user select portions of a given video file toprint, based on the histogram computation.

Selecting the Face Detection button 504 displays a dialog showingsegments along a timeline that contain face images. Each segment ispreferably accompanied by an integer that expresses the number of facesdetected in the clip as well as a confidence value. Slider controls arepreferably provided that let the user select portions of a given videofile to print, based the confidence values. In another embodiment, theface images are clustered so that multiple instances of the same faceare merged into one representation face image.

Selecting the Face Recognition button 522 presents a dialog showingnames along a timeline that were derived by application of facerecognition to video frames at corresponding points along the time line.Slider controls are preferably provided that let the user selectportions of a given video file to print. Also, a series of checkboxesare provided that let the user select clips by choosing names.

Selecting the OCR button 512 causes each frame in the video to be OCR'dand subsampled, for example once every 30 frames, and the resultsdisplayed along a timeline. Slider controls are preferably provided tolet the user select portions of a given video file to print based on theconfidence values that accompany the OCR results. A text entry dialog isalso preferably provided to let the user enter words that are searchedwithin the OCR results. Clips that contain the entered text areindicated along the timeline. In one embodiment, results of the OCRfunction are clustered so that similar OCR results are merged.

Selecting the Motion Analysis button 506 displays a dialog showing theresults of applying a motion analysis algorithm along a timeline. Onetype of motion analysis that can be used is a waveform having amagnitude that indicates the amount of detected motion. This allows auser to quickly locate the portions of a video that contain a personrunning across the camera's view, for example. A slider controlpreferably lets the user select portions of a given video file to print,based on the amount of motion that was detected.

Selecting the Distance Estimation button 510 presents a dialog showingthe results of applying a distance estimation algorithm along atimeline. For example, in a surveillance camera application using twocameras separated by a known distance, the distance of each point fromthe camera can be estimated. A slider control preferably lets the userselect portions of a given video file to print, based on their distancefrom the camera. For example, the user may wish to see only objects thatare more than 50 yards away from the camera.

In one embodiment, the motion analysis algorithm and distance estimationalgorithm are applied together.

Selecting the Foreground/Background Segmentation button 514 displays adialog showing the results of applying a foreground/backgroundsegmentation algorithm along a timeline. At each point, the foregroundobjects are displayed. A clustering and merging algorithm can be appliedacross groups of adjacent frames to reduce the number of individualobjects that are displayed. Slider controls are preferably provided tolet the user select portions of a given video file to print based on theconfidence value of the foreground/background segmentation as well asthe merging algorithm.

Selecting the Scene Segmentation button 518 presents a dialog showingthe results of applying a shot segmentation algorithm along a timeline.Each segment can be accompanied by a confidence value that thesegmentation is correct. A slider control is preferably provided to letthe user select portions of a given video file to print, based on theconfidence value.

Selecting the Visual Inspection button 528 presents a dialog showing animage from an attached video camera. The user can outline areas of thescene and define parameters for the objects that could appear in thoseareas. For example, for circular objects, a slider control preferablylets a user choose the diameter and allowable variations. This can beapplied to automatic inspection of objects on an assembly line such asball bearings that should be perfectly circular with a diameter of 2.54centimeters. The user can also choose actions that are executed when theselected parameters are exceeded. For example, a frame image should begrabbed and printed if a ball bearing is detected with a diameterdifferent from the defined value by more than 0.01 centimeters. Also, anemail can be sent to a specified user.

In one embodiment, scene segmentation can be combined with facerecognition. In such an embodiment, each segment can be accompanied by aconfidence value that the segmentation is correct. The results of facerecognition are shown as names along the timeline. Slider controls arepreferably provided to let the user select portions of a given videofile to print, based the confidence values of shot segmentation and facerecognition. Additionally, a series of checkboxes are provided to letthe user select clips by choosing names.

In another embodiment, scene segmentation can be combined with facedetection such that color or a special icon indicates segments on thetimeline that contain face images. Each segment can be accompanied by aninteger that expresses the number of faces detected as well as aconfidence value. Slider controls are preferably provided to let theuser select portions of a given video file to print, based theconfidence values of shot segmentation and face detection.

In another embodiment, scene segmentation and OCR can be combined. Insuch an embodiment, a dialog shows the results of applying a shotsegmentation algorithm along a timeline. Each segment can be accompaniedby a confidence value that the segmentation is correct. Each frame inthe video is OCR'd and subsampled, for example once every 30 frames, andthe results are displayed along the timeline. A text entry dialog isalso provided that lets the user enter words to be searched within theOCR results. Clips that contain the entered text are indicated along thetimeline. Slider controls are preferably provided to let the user selectportions of a given video file to print based on the confidence valuesthat accompany the shot segmentation and OCR results. In anotherembodiment, names are also shown on the timeline that were derived byapplication of face recognition to video frames. Additionally, a seriesof checkboxes are provided that let the user select clips by choosingnames. Slider controls are provided that let the user select portions ofa given video file to print based on the confidence values thataccompany the shot segmentation, OCR, and face recognition results.

In another embodiment, scene segmentation, OCR and face detection arecombined. In that embodiment, segments on the timeline that contain faceimages are shown. Each segment can be accompanied by an integer thatexpresses the number of faces detected in the clip as well as aconfidence value. Slider controls are preferably provided to let theuser select portions of a given video file to print based on theconfidence values that accompany the shot segmentation, OCR, and facedetection results.

Selecting the Automobile Recognition button 516 displays a dialog forconfiguring and using the automobile recognition function. In oneembodiment, a user operates a surveillance camera that creates manyhours of video, most of which is not of interest to the user. The userneeds to find and print only those sections that contain a specificobject, for example a red Cadillac. For this purpose, each frame in thevideo is input to an automobile recognition technique and the resultsare displayed along a timeline. Slider controls are preferably providedthat let the user select portions of a given video file to print basedon the confidence values that accompany the automobile recognitionresults. A text entry dialog is also preferably provided that lets theuser enter identifiers for the make, model, color, and year for anautomobile that are searched within the automobile recognition results.Clips that contain the entered information are indicated along thetimeline.

In another embodiment, a user often needs to find and print only thosesections that contain a specific license plate number. For this purpose,a user can select the License Plate Recognition button 520. Each framein the video is input to a license plate recognition technique and theresults, typically a plate number, state, plate color, name and addressof plate holder, outstanding arrest warrants, criminal history of theplate holder, etc., are displayed along a timeline. Slider controls arepreferably provided to let the user select portions of a given videofile to print based on the confidence values that accompany the licenseplate recognition results. A text entry dialog is also provided thatlets the user enter identifiers for the plate number, state, and year,etc. for a license plate that are searched within the license platerecognition results. Clips that contain the entered information areindicated along the timeline.

In another embodiment, a user needs to find and print only thosesections that contain a specific object, such as a speeding redCadillac. The automobile recognition function can be combined withmotion analysis for this purpose. Each frame in the video is preferablyinput to an automobile recognition technique and the results aredisplayed along a timeline. Additionally, a motion analysis technique isapplied to the video to estimate the automobile's speed from one frameto the next. Slider controls are preferably provided that let the userselect portions of a given video file to print based on the confidencevalues that accompany the automobile recognition and motion analysisresults. A text entry dialog is also provided that lets the user enteridentifiers for the make, model, color, and year for an automobile andits speed that are searched within the automobile recognition and motionanalysis results. Clips that contain the entered information areindicated along the timeline.

After the user has selected one or more multimedia processing functionsto apply, printer 100 performs the selected functions and produces apreview of the result. FIG. 6 illustrates an example of a previewedresult. In FIG. 6, display 302 shows a preview 602 of the output thatwill be printed, as well as a Continue button 314, a Cancel button 312and a Back button 604. If the user does not like the previewed output602, he can select the Back button 604, which in a preferred embodimentreturns the user to the previous screen. Alternatively, the user canpress the Cancel button 312 to cancel the print operation completely. Ifthe user wants to continue with the print job, he selects the Continuebutton 314 to go on. As those of skill in the art will recognize, theparticular preview 602 depends on the features that were applied. In theillustrated example of FIG. 6, sound localization and motion detectionhave been employed to create the displayed page.

Once the user is satisfied with the previewed output 602 and selects theContinue button, the next task in a preferred embodiment is to selectthe output path. In a preferred embodiment, output can be directed alonga printed output path 160, an electronic output path 170, or both.

Referring now to FIG. 7, there is shown an example of a display 302 of auser interface 110 with which a user can select an output path. Byselecting the Printed Output button 702, data will be routed to theprinted output system 115 and be printed in a conventional manner byprinter 100. If the user selects one of the buttons associated with theelectronic output system 120, data will be routed to the particularoutput device selected. In the illustrated case of FIG. 7, the user canselect a DVD-ROM drive 706, an Audio Cassette 708, VHR 710, OpticalStorage 712, Flash Memory 714, or Bluetooth device 716. Once the userhas made his selection, he presses the Print button 704 to send theoutput to the selected locations, or Cancel 312 to quit.

The present invention has been described in particular detail withrespect to a limited number of embodiments. Those of skill in the artwill appreciate that the invention may additionally be practiced inother embodiments. First, the particular naming of the components,capitalization of terms, the attributes, data structures, or any otherprogramming or structural aspect is not mandatory or significant, andthe mechanisms that implement the invention or its features may havedifferent names, formats, or protocols. Further, the system may beimplemented via a combination of hardware and software, as described, orentirely in hardware elements. Also, the particular division offunctionality between the various system components described herein ismerely exemplary, and not mandatory; functions performed by a singlesystem component may instead be performed by multiple components, andfunctions performed by multiple components may instead performed by asingle component. For example, the particular functions of the userinterface 110 and so forth may be provided in many or one module.

Some portions of the above description present the feature of thepresent invention in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are the means used by those skilled in the userinterface and printing arts to most effectively convey the substance oftheir work to others skilled in the art. These operations, whiledescribed functionally or logically, are understood to be implemented bycomputer programs. Furthermore, it has also proven convenient at times,to refer to these arrangements of operations as modules or code devices,without loss of generality.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the present discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system memories or registersor other such information storage, transmission or display devices.

Certain aspects of the present invention include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the present inventioncould be embodied in software, firmware or hardware, and when embodiedin software, could be downloaded to reside on and be operated fromdifferent platforms used by real time network operating systems.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but is not limited to, any type ofdisk including floppy disks, optical disks, CD-ROMs, magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, application specific integratedcircuits (ASICs), or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus. Furthermore,the computers referred to in the specification may include a singleprocessor or may be architectures employing multiple processor designsfor increased computing capability.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may also be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear from the description above.In addition, the present invention is not described with reference toany particular programming language. It is appreciated that a variety ofprogramming languages may be used to implement the teachings of thepresent invention as described herein, and any references to specificlanguages are provided for disclosure of enablement and best mode of thepresent invention.

Finally, it should be noted that the language used in the specificationhas been principally selected for readability and instructionalpurposes, and may not have been selected to delineate or circumscribethe inventive subject matter. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting, of the scopeof the invention.

1. A multifunction printer for printing time-based media, themultifunction printer comprising: a communication interface forreceiving time-based media data from a media source; a user interfacedisplay on the multifunction printer that displays a function selectionmenu comprising a plurality of user-selectable audio and video rangeselection functions to be applied to the time-based media by a processorwithin the multifunction printer; an input device for receiving from auser a selection of a multimedia function from the plurality ofuser-selectable video and audio range selection functions displayed onthe function selection menu; the processor within the multifunctionprinter for performing the multimedia function selected from theplurality of user-selectable video and audio range selection functionsand for generating a printable representation comprising results of themultimedia function applied to the time-based media; a first outputdevice for receiving the printable representation from the processor andfor automatically printing the printable representation to a tangiblemedium; and a second output device coupled to the processor forreceiving the printable representation from the processor and foroutputting an electronic output including the printable representation.2. The printer of claim 1 wherein the multimedia function includesinstructions for selecting a range of audio data in response to the userselecting one of the audio range selection functions from the functionselection menu.
 3. The printer of claim 1 wherein the multimediafunction includes instructions for applying audio event detection to thetime-based media data.
 4. The printer of claim 3 wherein the multimediafunction further includes instructions for determining a confidencelevel associated with the audio event detection.
 5. The printer of claim1 wherein the multimedia function includes instructions for applying aspeaker segmentation function to the time-based media data.
 6. Theprinter of claim 1 or 5 wherein the multimedia function includesinstructions for applying a speaker recognition function to thetime-based media data.
 7. The printer of claim 1 wherein the multimediafunction includes instructions for applying a sound source localizationfunction to the time-based media data.
 8. The printer of claim 7 whereinthe multimedia function further includes instructions for applying audioevent detection to the time-based media data.
 9. The printer of claim 1wherein the multimedia function includes instructions for applying aspeech recognition function to the time-based media data.
 10. Theprinter of claim 9 wherein the multimedia function includes instructionsfor applying a profile analysis function to the time-based media data.11. The printer of claim 9 wherein the multimedia function includesinstructions for applying an audio event detection function to thetime-based media data.
 12. The printer of claim 11 wherein themultimedia function further includes instructions for applying a speakerrecognition function to the time-based media data.
 13. The printer ofclaim 11 wherein the multimedia function further includes instructionsfor applying a speaker segmentation function to the time-based mediadata.
 14. The printer of claim 11 wherein the multimedia functionfurther includes instructions for applying a sound localization functionto the time-based media data.
 15. The printer of claim 1 wherein themultimedia function includes instructions for selecting a range of videodata in response to the user selecting one of the audio range selectionfunctions from the function selection menu.
 16. The printer of claim 1wherein the multimedia function includes instructions for applying avideo event detection function to the time-based media data.
 17. Theprinter of claim 1 wherein the multimedia function includes instructionsfor applying a color histogram analysis function to the time-based mediadata.
 18. The printer of claim 1 wherein the multimedia functionincludes instructions for applying a face detection function to thetime-based media data.
 19. The printer of claim 18 wherein themultimedia function includes instructions for applying a clusteringfunction to the time-based media data to merge multiple instances of aface into a representative face image.
 20. The printer of claim 1wherein the multimedia function includes instructions for applying aface recognition function to the time-based media data.
 21. The printerof claim 1 wherein the multimedia function includes instructions forapplying an optical character recognition function to the time-basedmedia data.
 22. The printer of claim 21 wherein the multimedia functionfurther includes instructions for applying a clustering function to thetime-based media data to merge similar results of the optical characterrecognition.
 23. The printer of claim 1 wherein the multimedia functionincludes instructions for applying a motion analysis function to thetime-based media data.
 24. The printer of claim 1 or claim 23 whereinthe multimedia function includes instructions for applying a distanceestimation function to the time-based media data.
 25. The printer ofclaim 1 wherein the multimedia function includes instructions forapplying foreground/background segmentation function to the time-basedmedia data.
 26. The printer of claim 1 wherein the multimedia functionincludes instructions for applying a scene segmentation function to thetime-based media data.
 27. The printer of claim 26 wherein themultimedia function further includes instructions for applying a facerecognition function to the time-based media data.
 28. The printer ofclaim 26 wherein the multimedia function further includes instructionsfor applying a face detection function to the time-based media data. 29.The printer of claim 26 wherein the multimedia function includesinstructions for applying an optical character recognition function tothe time-based media data.
 30. The printer of claim 29 wherein themultimedia function further includes instructions for applying a facerecognition function to the time-based media data.
 31. The printer ofclaim 29 wherein the multimedia function includes instructions forapplying a face detection function to the time-based media data.
 32. Theprinter of claim 1 wherein the multimedia function includes instructionsfor applying an automobile recognition function to the time-based mediadata.
 33. The printer of claim 32 wherein the multimedia functionfurther includes instructions for applying a motion analysis function tothe time-based media data.
 34. The printer of claim 1 wherein themultimedia function includes instructions for applying a license platerecognition function to the time-based media data.
 35. The printer ofclaim 1 wherein the multimedia function includes instructions forapplying a visual inspection function to the time-based media data. 36.The printer of claim 1 wherein the user interface display is configuredto allow a user to control a compact disc (CD) device.
 37. The printerof claim 1 wherein the user interface display is configured to allow auser to control a digital video disc (DVD) device.
 38. The printer ofclaim 1 wherein the user interface display is configured to allow a userto control an audio tape device.
 39. The printer of claim 1 wherein theuser interface display is configured to allow a user to control a videotape device.
 40. The printer of claim 1 wherein the user interfacedisplay is configured to allow a user to control a multimedia server.41. The printer of claim 1 wherein the user interface display isconfigured to allow a user to control encryption hardware.
 42. Theprinter of claim 1 wherein the user interface display is configured toallow a user to control audio sound localization hardware.
 43. Theprinter of claim 1 wherein the user interface display is configured toallow a user to control motion detection hardware.
 44. The printer ofclaim 1 wherein the user interface display is configured to allow a userto control a MIDI player.
 45. The printer of claim 1 wherein the userinterface display is configured to allow a user to control a cellulartelephone.
 46. The printer of claim 1 wherein the user interface displayis configured to allow a user to control a two-way radio.
 47. Theprinter of claim 1 wherein the user interface display is configured toallow a user to control a world wide web display.
 48. The printer ofclaim 1 wherein the user interface display is configured to allow a userto control a climate sensor.
 49. The printer of claim 1 wherein the userinterface display is configured to allow a user to control a radioreceiver.
 50. The printer of claim 1 wherein the second output device isa DVD drive.
 51. The printer of claim 1 wherein the second output deviceis a CD drive.
 52. The printer of claim 1 wherein the second outputdevice is an audio tape drive.
 53. The printer of claim 1 wherein thesecond output device is a video cassette device.
 54. The printer ofclaim 1 wherein the second output device is a removable media device.55. The printer of claim 1 wherein the second output device is anembedded audio recorder.
 56. The printer of claim 1 wherein the secondoutput device is an embedded video recorder.
 57. The printer of claim 1wherein the second output device is a non-volatile storage device. 58.The printer of claim 1 wherein the second output device is an embeddedmultimedia server.
 59. The printer of claim 1 wherein the second outputdevice is audio encryption hardware.
 60. The printer of claim 1 whereinthe second output device is video encryption hardware.
 61. The printerof claim 1 wherein the second output device is audio sound localizationhardware.
 62. The printer of claim 1 wherein the second output device isa cellular telephone.
 63. The printer of claim 1 wherein the secondoutput device is a two-way radio.
 64. The printer of claim 1 wherein thesecond output device is a world-wide web display.
 65. The printer ofclaim 1 wherein the second output device is a radio receiver forreceiving AM signals.
 66. The printer of claim 1 wherein the secondoutput device is a radio receiver for receiving FM signals.
 67. Theprinter of claim 1 wherein the second output device is a radio receiverfor receiving short wave signals.
 68. The printer of claim 1 wherein thesecond output device is a satellite radio receiver.
 69. The printer ofclaim 1 wherein the second output device is a weather alert receiver.70. The printer of claim 1 wherein the second output device is anemergency alert monitor for receiving emergency broadcast system alerts.71. The printer of claim 1 wherein the second output device is hardwarefor performing VGA screen captures.
 72. The printer of claim 1 whereinthe second output device is hardware for performing audio capture. 73.The printer of claim 1 wherein the second output device is hardware forcapturing data from an electronic pen.
 74. The printer of claim 1wherein the second output device is a disposable media writer.
 75. Theprinter of claim 1 wherein the second output device is a flash memorydevice.
 76. The printer of claim 1 wherein the second output device is awireless device.
 77. The multifunction printer of claim 1, wherein theuser interface display on the multifunction printer further displays aninput source selection menu comprising a plurality of user-selectableinput sources, wherein the input device further receives a selection ofan input source from the plurality of user-selectable input sources, andwherein the communication interface receives the time-based media datafrom the input source selected by the user.
 78. The multifunctionprinter of claim 1, wherein the user interface display on themultifunction printer further displays an output source selection menucomprising a plurality of user-selectable electronic output sources,wherein the input device further receives a selection of an electronicoutput source from the plurality of user-selectable output sources, andwherein the second output device outputs the electronic output to theelectronic output source selected by the user.
 79. The multifunctionprinter of claim 1, wherein the user interface display outputs thefunction selection menu responsive to the user selecting the inputsource using the input source selection menu.
 80. The multifunctionprinter of claim 1, wherein the user interface display outputs theoutput source selection menu responsive to the user selecting themultimedia function using the function selection menu.
 81. Themultifunction printer of claim 1, wherein the user interface display onthe multifunction printer further displays a sub-menu for displayingoptions to the user specific to the user-selected multimedia function,wherein the sub-menu is displayed responsive to the user selecting oneof the video range selection functions and the audio range selectionfunctions from the function selection menu.
 82. The multifunctionprinter of claim 1, wherein the user interface display on themultifunction printer further displays a preview menu for displaying apreview of the printable representation to the user responsive to theuser selecting the multimedia function using the function selectionmenu.
 83. The multifunction printer of claim 82, wherein the previewincludes a timeline of the multimedia data indicating results ofapplying the multimedia function to the time-based media data along thetimeline.
 84. The multifunction printer of claim 83, wherein the previewincludes user-selectable options for refining parameters of themultimedia function, the user interface display updating the timelineresponsive to the user changing the parameters.
 85. The multifunctionprinter of claim 1, wherein the plurality of user-selectable audio rangeselection functions on the function selection menu displayed by the userinterface display on the multifunction printer comprises at least one ofan event detection function, a speaker segmentation function, a speakerrecognition function, a sound source location function, a speechrecognition function, and a profile analysis function.
 86. Themultifunction printer of claim 1, wherein the plurality ofuser-selectable video range selection functions on the functionselection menu displayed by the user interface display on themultifunction printer comprises at least one of an event detectionfunction, a color histogram analysis, a face detection function, a facerecognition function, an optical character recognition function, amotion analysis function, a distance estimation function, aforeground/background segmentation function.
 87. The multifunctionprinter of claim 1, wherein the user interface display further displaysa user-selectable print function for printing the printablerepresentation, and wherein the first output device prints the printablerepresentation responsive to the input device receiving a selection ofthe print function from the user.
 88. A method for printing time-basedmedia, the method comprising: receiving time-based media data from amedia source; displaying, on a user interface display of a multifunctionprinter, a function selection menu comprising a plurality ofuser-selectable audio and video range selection functions to be appliedto the time-based media; receiving a user selected multimedia function,wherein the user selected multimedia function is a function selectedfrom the plurality of user-selectable audio range and video rangeselection functions displayed on the function selection menu;performing, by the multifunction printer, the user selected multimediafunction on the time-based media data; generating, by the multifunctionprinter, a printable representation comprising results of the userselected multimedia function applied to the time-based media; printingthe printable representation to a tangible medium; and outputting anelectronic output of the printable representation.
 89. The method ofclaim 88 wherein the multimedia function includes instructions forselecting a range of audio data in response to a user selecting one ofthe audio range selection functions from the function selection menu.90. The method of claim 88 wherein the multimedia function includesinstructions for applying audio event detection to the time-based mediadata.
 91. The method of claim 90 wherein the multimedia function furtherincludes instructions for determining a confidence level associated withthe audio event detection.
 92. The method of claim 88 wherein themultimedia function includes instructions for applying a speakersegmentation function to the time-based media data.
 93. The method ofclaim 88 or 92 wherein the multimedia function includes instructions forapplying a speaker recognition function to the time-based media data.94. The method of claim 88 wherein the multimedia function includesinstructions for applying a sound source localization function to thetime-based media data.
 95. The method of claim 94 wherein the multimediafunction further includes instructions for applying audio eventdetection to the time-based media data.
 96. The method of claim 88wherein the multimedia function includes instructions for applying aspeech recognition function to the time-based media data.
 97. The methodof claim 96 wherein the multimedia function includes instructions forapplying a profile analysis function to the time-based media data. 98.The method of claim 96 wherein the multimedia function includesinstructions for applying an audio event detection function to thetime-based media data.
 99. The method of claim 98 wherein the multimediafunction further includes instructions for applying a speakerrecognition function to the time-based media data.
 100. The method ofclaim 98 wherein the multimedia function further includes instructionsfor applying a speaker segmentation function to the time-based mediadata.
 101. The method of claim 98 wherein the multimedia functionfurther includes instructions for applying a sound localization functionto the time-based media data.
 102. The method of claim 88 wherein themultimedia function includes instructions for selecting a range of videodata in response to a user selecting one of the audio range selectionfunctions from the function selection menu.
 103. The method of claim 88wherein the multimedia function includes instructions for applying avideo event detection function to the time-based media data.
 104. Themethod of claim 88 wherein the multimedia function includes instructionsfor applying a color histogram analysis function to the time-based mediadata.
 105. The method of claim 88 wherein the multimedia functionincludes instructions for applying a face detection function to thetime-based media data.
 106. The method of claim 105 wherein themultimedia function includes instructions for applying a clusteringfunction to the time-based media data to merge multiple instances of aface into a representative face image.
 107. The method of claim 88wherein the multimedia function includes instructions for applying aface recognition function to the time-based media data.
 108. The methodof claim 88 wherein the multimedia function includes instructions forapplying an optical character recognition function to the time-basedmedia data.
 109. The method of claim 108 wherein the multimedia functionfurther includes instructions for applying a clustering function to thetime-based media data to merge similar results of the optical characterrecognition.
 110. The method of claim 88 wherein the multimedia functionincludes instructions for applying a motion analysis function to thetime-based media data.
 111. The method of claim 88 or claim 110 whereinthe multimedia function includes instructions for applying a distanceestimation function to the time-based media data.
 112. The method ofclaim 88 wherein the multimedia function includes instructions forapplying foreground/background segmentation function to the time-basedmedia data.
 113. The method of claim 88 wherein the multimedia functionincludes instructions for applying a scene segmentation function to thetime-based media data.
 114. The method of claim 113 wherein themultimedia function further includes instructions for applying a facerecognition function to the time-based media data.
 115. The method ofclaim 113 wherein the multimedia function further includes instructionsfor applying a face detection function to the time-based media data.116. The method of claim 113 wherein the multimedia function includesinstructions for applying an optical character recognition function tothe time-based media data.
 117. The method of claim 116 wherein themultimedia function further includes instructions for applying a facerecognition function to the time-based media data.
 118. The method ofclaim 116 wherein the multimedia function includes instructions forapplying a face detection function to the time-based media data. 119.The method of claim 88 wherein the multimedia function includesinstructions for applying an automobile recognition function to thetime-based media data.
 120. The method of claim 119 wherein themultimedia function further includes instructions for applying a motionanalysis function to the time-based media data.
 121. The method of claim88 wherein the multimedia function includes instructions for applying alicense plate recognition function to the time-based media data. 122.The method of claim 88 wherein the multimedia function includesinstructions for applying a visual inspection function to the time-basedmedia data.